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Input devices 
A 
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202 



204 



206 



208 



1 



Microphone 



Video camera 



Body temp. 
sensor 



EKG/EEG 
sensor 



Heart rate 
monitor 



Blood pressure 
monitor 



210 Blood 0 2 sensor 



212 
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Ambient temp, 
sensor 



Various user 
sensor devices 
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Various 
environment 
sensor devices 
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Various user 
input devices 



CPU 



■ 1 ^ 



120 



t T V 



234 

232 




248 



214 



Audio input 
CS 



216 



Video input CS 



218 



Body temp. CS 



220 



Heart rate CS 



222 



Blood pressure 
CS 



224 



Blood Ch CS 



226 



Other CS's 



230 

A 



CM 



240 



Formatter 
and visual 
display 
selector 



242 

A 



Z =3.2 

O 
O 



244 



Other CS's n 



Application program 



236 



Browser program 



238 



Fig. 2 



Output devices 
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Head-mounted 
display 



130 



Flat panel 
display 



132 



Earpiece 
speaker 



136 



Tactile output 
device 



252 



Telecom, 
transceiver 



254 



Speaker 



250 



Defibrillator 
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Various user 
output devices 



Code 
module A 
(CS) 



Code 
module B 
(CC) 



Code 
module C 
(CS/CC) 



Code 
module D 
(CS/CC) 
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Initialize CS and/or CC 
for theme set for user 



402 



At CS, obtain data signals from input devices 



At CS, process data signals to produce 
attributes representing user's context, 
including adding required data fields, such as 
attribute name, timestamp, units, etc. 



At CS, provide attribute to CM for storage 



At CC, request from CM and receive attributes 



404 



406 



408 



410 



At CC, process attributes to determine appropriate output 



Provide output signal to user, other entity or other process 



412 
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Fig. 4 



Launch at start-up 
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Periodically receive GPS data stream 



504 



Parse and store GPS data 



506 



508 

Receive 
request for latitude^v^ N 
^ongitude or altitude, 
^attribute?^ 



Provide requested attribute value(s), 
with Timestamp, uncertainty, units, etc. 



51 



Fig. 5 A 



Name 

Latitude (value) 



Uncertainty 



Timestamp 



Units 



Format version/flags 



521 
522 
524 
526 
528 
530 



Fig. 5B 



Register for latitude, longitude and altitude attributes 



602 



604 




Normal operation? 



606 



Use default attribute mediator 



Reevaluate all latitude attributes 



608 



Select lowest uncertainty attribute mediator 



Determine CS providing resulting attributes 



610 



612 



Request latitude, longitude and altitude attributes from CS/CM 



616 




Receive input derfining boundary 



•618 



Provide event parameters to CM regarding boundary transversal 



622 

Receive 
response from CM to^>N 
^established event? 



Notify user that boundary has been crossed 
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Fig. 6 
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Desired_privacy_level 

Interruptibility 

Speed 

Direction 

Acceleration 

Availability. 

Cognitive_availability 

Tactile_availability 

Manual_availability 

Visual_availability 
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Auditory_availability 
Proximity. <Item or place name> 
Mood. 

Happiness 
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Anger 

Frustration 
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Activity. 

Driving 

Eating 

Running 

Sleeping 

Talking 

Typing 

Walking 
Location. 

Place_name 

Latitude 

Longitude 

Altitude 

Room 

Floor 

Building 

Address 

Street 

City 

County 

State 

Country 

Postal_Code 
Destination, (same as User. Location.) 
Physiology. 

Pulse 

Body_temperature 

Blood_pressure 

Respiration 
Person. <name or ED>. (same as User.) 
Platform. 
UI. 

Oral_input_device_a vailabil ity 
Manual_input_device_a vailabil ity 
Tactile_output_device_availability 
Vi sual_output_device_a vai labil ity 
Auditory_output_device_availability 
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Platform, (continued) 
CPU 

Load 

Speed 
Memory. 

Total_capacity 

Used 
Storage. 

Total_capacity 
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Connectivity. 

Connection_status 

Connection_speed 

Connection_type/device 

Connection_activity 
Power. 
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Sound_recorder. 
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Recording 
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Scope of Disclosure 
Hardware affinity for privacy 
Privacy Indicator for User 
Privacy Indicator for Public 
Watching Indicator 
Being Observed Indicator 
Ambient Interference 
Visual 
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Example Current Theme Selection 



1150 



/-Starting The Day 

-Driving Home 
-At Home 

Work-Related Group 
'Driving To Work 
-Arriving At Work 
-At Work 




Select A New Current Theme 



1155 



Entertainment-Related Group 



OK 



1157 



Cancel 
J 



Figure 11M 



Starting The Day 

1150 ^ At Home 

. Eating Breakfast 



Example Current Theme Set Modification 
Current Theme Set 



Add Theme 



1157 



A. 



Remove Theme 



Figure UN 



Example Current Theme Layout Selection 

Select A New Current Theme Layout 

Starting The Day theme 
r Starting The Day - default layout 
1160 < Starting The Day - vacation 

L Starting The Day - responsible for dropping off kids 



OK 



Cancel 



1157 



Figure HO 





1302 

s 


1304 

\ Example Theme Data Structure 
) 1300 


1305^X> 


Name 


Driving To Work 


1310'-\> 


Description 


Intent Is To Organize Both Driving Info & Info About Work Arrival 


1315^\_, 


Security 


Default: ***♦; Self: ******;... 


1320^\> 


Permission 


Access: Self + Family + Friends + Source; Modification: Self; Copy: None; . . . 


1325'~v> 


Privacy 


Private 


1330^\^ 


Priority 


Medium 


1335^\> 


Group-ID 


23 


1340^X> 


Time-Active 


Mon-Fri, 06:00-11:00 


1345'~^ 


Source 


BCD Motor Company 


1350^^ 


Theme-Content 


"BCD Motor logo.gif ' 


1 1CC 


New-Attributes 


DTW-BCD.driving-direction: (source = "BCD-DTW CS"); . . . 


1360^^ 


Theme-Matching 


user.activity.driving - true (required); environment.local.time = 08:00-09:00 (optional); 
DTW-BCD.driving-direction = NOT "South" (required); . . . 


1365^, 


Theme-Logic 


IF <user.activity.driving> AND user.location.latitude > recent.user. location.latitude 

THEN DTW-BCD.driving-direction = "North"; 
IF DTW-BCD.driving-direction = (< East" THEN AlertfYou're Lost Again! !"); 
SET user, location [Access] = Family & Co-workers; . . . 


1370^^ 


Attribute-Set 


<linki>; • ♦ • <link v > 




; i \ 



r Attribute 1 



1390 < 



Name 


user.activity.driving 


Value 


True 


Uncertainty 


High 


Timestamp 


07:32 03/24/XX 


Units 


boolean 


Access 


Family 





Attribute N 



Name 


DTW-BCD.driving-direction 


Value 


West 


Uncertainty 


None 


Timestamp 


08:13 03/24/XX 


Units 


North, North by Northeast, . . . 


Access 


Self 





Fig. 13 
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1500 

(Theme usage ^ 
routine J 



1505 



Receive indications of currently available themes 
and their associated theme information 




r 1510 


Retrieve current c< 


)ntext information 


► 


r 1515 


Determine themes that match current context 



1530 



Present matching or all 


< Yes, 


themes to user 






r 1535 




Receive indication of 




selected theme 





1520 



User selection? 



No 



1525 



Select highest priority 
matching theme 



1540 



Generate appropriate 
response to selected theme 



1545 



Detect changes to context or user indication 



1555 



Retrieve context changes 



1550 



No, 



User indication? 



1565 



Present current context 
information to user 





Yes 


1 


r 1560 



Yes. 



Change context 
information? 



.No 




1595 



Q END ^ 



Yes 1575 



Exit? 



No 



1570 



1580 



Receive indication of 
context changes 



Provide indicated 
functionality to user 



Fig. 16 



1615 



Select default theme layout 
if multiple are associated 



1540 

Response generator" 
subroutine 



1 


r 1605 


Receive indication of selected theme 
and of associated theme information 




r 1610 



Yes, 



Associated theme 
layout(s)? 



.No 



1620 



Retrieve content specified 
by theme layout 



1625 1630 



Perform data logic specified 
by theme layout 




Retrieve user preference 
information 


► 



± 1635 

Present content in manner specified by presentation logic 
of theme layout and in a manner consistent with 
preference information 



1640 



Other response 
specified by theme 
layout? 



i 1645 

Provide other response as 
appropriate 



1695 



^ Return ^ 



1700 

(Theme creator/ ^\ 
modifier routine J 



n 

m 

w 
m 

N 

m 



w 

Pi 











r 1705 




Receive indication of theme 
to be created or modified 


1715 




r 1710 



Fig. 17 



1720 




Display theme to user 



1730 



Receive indication from user to modify 
specified properties of theme or 
information associated with theme 



1740 



Present theme layout to user 
in visual editor 



1735 



Modify theme 
layout? 





No 




r 1750 



1755 



Modify property? 



,Yes 



1745 



Modify theme layout as 
indicated by user 



Modify theme property as 
indicated by user 



No 



1760 



Modify theme attribute, 
theme CS, or theme CC as 
indicated by user 



1765 



Yes, 



More changes to 
theme? 



No 



1770 



Store theme and 
theme-related information 



Yes y 



v 1775 



More themes? 



1795 



No 



< end ) 



1800 

/Theme distributor^ 
I routine ) 



Fig. 18 



1805 



Receive indications of 
available themes 



1810 



Retrieve indications of 
appropriate users for the 
available themes 



1815 



Receive request for theme(s) 
or indication of accessible 
user 



1835 



Retrieve information about 
user from database or user's 
computing device 




1820 



1825 



Request? 



.Yes 



Determine if user has 
provided appropriate access 
information 



1830 



1840 



Determine themes that are 
appropriate for user 



f 



Yes, 



Appropriate 
access? 



1845 




Determine if any needed 
payment information has 
been provided 



1850 



Payment 
provided? 



.No 



Yes 



1855 



Gather theme-related information 
associated with requested or determined 
themes, including the themes 



1860 



Send gathered information 
to requesting or indicated 
user 



1865 



Yes, 



Continue? 





No 


1 


r 1895 



1900 

(Theme receiver^ 
routine J 



1905 



Receive user request for 
theme or indication of sent 
theme 



No y 



1 


r 1940 


Receive indication of sent 
theme 




r 1945 



No, 



Payment 
information 
needed? 





Yes 

r 1950 


Send payment information 




r 1955 



Payment accepted? ;>^° 



Yes 



1960 



Receive theme and any 
associated theme-related 
information 



1965 



Store received information, 
and load for use if 
immediately useful 



Yes. 



Fig. 19 



v 1910 



1915 



Request? 



,Yes 



v 1922 



Indicate failure 
to user 



v 1970 



Continue? 



Attempt to determine theme 
server that can provide 
requested theme 



1920 



No. 



Theme server 
found? 



Yes 



1925 



Receive appropriate access 
information for theme and 
theme server, either from 
user or from storage 



1930 



Receive indication of 
payment mechanism if 
needed, either from user or 
from storage 




r 1935 


Send request 
theme server, 
access infor 
payment ii 


for theme to 
including any 
mation and 
lformation 





No 




r 1995 



END 



2000 

/Automated theme\ 
\customizer routine J 



2005 



Receive indications of available themes 



± 2010 

Repeatedly monitor user actions, 
including interactions with themes, 
modifications to themes, and explicit 
changes to the current theme or the 
current context 



Fig. 20 



2015 



Analyze monitored actions to determine 
patterns of actions 




r 2020 



No 



Any patterns above relevance 
threshold? 



1 


Yes 

r 2025 


Retrieve user preference information 




r 2030 


Determine modifications to some or all 
themes that are consistent with patterns 
above the relevance threshold and with 
user preferences 




r 2035 



2050 



Do automated modification? 



> 


Yes 

r 2040 


Modify themes as determined 




r 2045 


Notify user of modifications 




r 2065 



Yes 



Continue? 





No 


> 


r 2095 



No p 


Present suggested modifications to the 




user 




> 


r 2055 




Receive indications of accepted 
modifications 






r 2060 





Modify themes as indicated 





Q END ^ 




